Fusing Online and Offline Information for Stable 3D Tracking in Real-Time

We propose an efficient online real-time solution for singlecamera 3–D tracking of rigid objects that can handle large camera displacements, drastic aspect changes, and partial occlusions. While the offline camera registration problem can be considered as essentially solved, robust online tracking remains an open issue because many real-time algorithms described in the literature still lack robustness and are prone to drift and jitter. To solve these problems, we have developed a robust approach to 3–D feature matching that can handle widebaseline matching: our method merges the information from preceding frames in traditional recursive tracking fashion with that provided by a very limited number of keyframes created during an offline stage. This combination results in a system that does not suffer from the above difficulties and can deal with drastic aspect changes. We use Augmented Reality applications to demonstrate its behavior because they are particularly demanding in terms of tracking performance.